Papers with adversarial accuracy
Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation (D19-1)
Copied to clipboard
| Challenge: | Existing evaluation methods for natural language generation are inadequate . distinguishing machine-generated text is challenging even for human evaluators . |
| Approach: | They compare human-based evaluators with automated evaluation procedures . they find human evaluers do not correlate well with discriminative evalators . |
| Outcome: | The proposed evaluation methods are compared with a dozen state-of-the-art generators for online product reviews. |
Certified Robustness to Adversarial Word Substitutions (D19-1)
Copied to clipboard
| Challenge: | State-of-the-art NLP models can be fooled by adversaries that apply seemingly innocuous label-preserving transformations to input text. |
| Approach: | They propose to train models that are provably robust to all word substitutions in a family of label-preserving transformations that can be replaced with a similar word without changing the original sentiment. |
| Outcome: | The proposed models achieve 75% adversarial accuracy on both sentiment analysis and natural language inference on IMDB and SNLI compared to models trained normally and ones trained with data augmentation. |
Don’t Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Text (2023.acl-long)
Copied to clipboard
| Challenge: | ATINTER model can be used to rewrite adversarial inputs to make them non-adversarial . if undefended, model should maintain good task performance and effectively mitigate adversarials . |
| Approach: | They propose a model that intercepts adversarial inputs and learns to rewrite them . they show that it provides better adversarial robustness than existing defense approaches . |
| Outcome: | The proposed model improves adversarial robustness without compromising task accuracy on a sentiment classification dataset. |